Formalization of Multimodal Languages in Pervasive Computing Paradigm

The increasing need to access information everywhere and at any time leads us to believe that future user interfaces, through which users interact with pervasive computing systems, must enable both device and modality independence. The pervasive computing paradigm sees almost every object in the everyday environment as a system able to communicate with users and other systems in their own languages. The interaction between users and systems is therefore typically multimodal. In this paper, features and computational issues of the multimodal interaction are analyzed in order to examine methodological aspects for the definition of multimodal interaction languages and usable multimodal interfaces for pervasive applications. Its main objective is thus to explore multimodal interaction for pervasive applications through use of a multimodal language rather than through the integration of several unimodal languages.